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ABSTRACT 

The effect of testing on student retention '«as 
investigated to determine if it was the actual test taking, the prior 
preparation for the test, or the combination of studying for and 
taking the test which actually affected retention. Subjects were 
undergraduate students (N=90) who were divided into two different 
study conditions, i»e, , test and no-test. The test condition 
consisted of two different kinds of test, multi pie-ch'oice and 
short-answer, each . containing two different levels of item 
complexity, knowledge and concept- Results indicated that: (1) 
subjects in the study condition test group scored higher on retention 
than subjects in the stu^y condition no-test group: <2) study 
condition test subjects who tcck either the multiple-choice or 
fihort^answer- test scored higher than subjects who took no test; (3). 
means for the multiple-choice test and short- answer test groups did 
not significantly differ: and (U) the short-answer test treatment 
group failed to differ from the no-test treatment group, 
(Author/HLM) 
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Suimiary ' 

In 1885 Ebblnghaus published, the first expei^imental study of human 
memory. There have been numerous studies in the fields of education and 
psychology since that time which have addressed the problems of learning and 
retention. This research generally falls into two basic categories: 
(a) studies concerned with theory which focus on the recall of word pairs, 
series of words, sentences, and the like and (b) more practical research 
which includes studies of educational achievement- and the retention of 
learning. A survey of the literature revealed far more studies in the former 
category than in the latter. An even greater deficit was found in the number 
of studies of specific educational practices related to retention of classroom 
learning. Harnquist (1977) highlighted this deficiency in an invited address 
to the 1977 meeting of the American Educational Research Association which he' 
called "Enduring Effects of School--A Neglected Area in Educational Research." 

Our study is one in this neglected area. It was designed to investigate 
the effect of a particular classroom activity— testing— on student retention. 
More specificallv, the purpose of this study was' to determine if it is the 
actual taking of the test, the prior^preparation f )r the test, or a combination 
of studying for and taking the test which affects student retention if such 
an effect exists. Further purposes of this study were to determine If type of 
criterion measure (multiple-choice or short answer) or complexity of test item 
(knowledge or concept) significantly moderates the effects of testing on retention 

Method 

Subjects 

Subjects for this study were 90 undergraduate students enrolled in 
five educational psychology classes at a large southern university which 
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attracts students with diverse backgrounds from an extensive geographical , 
area. Included were both female (N « 60) and male (N « 30) education (N«^^62) 
and nonedMcation (N = 28) majors (median age = 20 years). 

' '' ( 

Procedure 

Treatment. ' Treatment in this experiment consisted in part of two 

I ' ■■ ■■» Ji 

different study conditions: test and no test. In the test condition, 
subjects read the text assignment and attended class with the expectation of " 
being tested. In the no test condition, subjects were asked to "read the 
•text assignment and attend class to learn rather than for a test." 

Treatment in this study further consisted of two different kinds of 
tests, multiple-choice (30 items) and short answer (38 items created from - 
the multiple-choice item stems)', each containing two different levels of 
item complexity, knowledge (15 .items) and concept (15 items). 

P.ppri»ntal Procedures. Using class rolls, subjects within each 
class w;re randomly assigned to one of six groups and each group was randomly 
assigned (a) study condition and (b) test treatments as follows: 
Group 1:- (a) test, (b) multiple-choice 
Group 2: (a) test, (b) short answer 
Group 3; (a) test, (b) no test 
Group 4: (a) no test, (b) multiple-choice 
Group 5; (a) no test, (b) shcrt answer 
Group 6: (a) no test, (b) no test 

During the Introductory meeting of each of the educational psychology 
classes students were given by their instructor a syllabus with assignments 
and test dat.s (study condition-test). Each instructor subsequently called 
the names of those students Who had been assigned to study condition-no test 



ana asked them; to stay briefly after cTass where .they met w^h an experimenter 
who explained that they had been selected to participate in an evaluation of ^ 
ongoing instructional methods. For tfieir j)articipation. which' they were not^ ' 
to'discuss with anyone, they would be given , art "A" in lieu of their earned . 
grade on the upcoming .test. They were, however. >til.l asked to "read the , 
textbook assignment and attend class but in orderto learn rather than for a 

test." ./ — ' ' ' , 

Each ins.tructor (white females) taught .behavioristic learning theory to 

her respective clays using a lecture-discussion W^oach for the'^next^two .- 
weeks. On the assigned test day two of the experimenters came to. the^ 
classroom and asked for by name those students who had been assigned to the ^ 
study condition-no test group as well as those in Group 3-3tudy condition- 
test, test condition-no test. ^ 

Subjects in Group 3 and Group 6-study condition-no test, test 
conditi6n-no test went with one experimenter to a .vacant classroom where 
they were £old.,as some of the group already knew, that theV class was - 
participatihgjn an evaluation of instructional techniques which would 
necessitate their not responding to thete^t their classmates were taking. 

Subjects in Group 4-study condition-no test, test condition-multiple-, 
choice and Group 5-study condition-no test, test condition-short answer went 
with another experimenter to a vacant classroom where it i^as explained ^to them 
that one of their functions in the evaluation project previously discussed 
was.to respond to the test their classmates were taking. All agreed to 
continue to participate and were administe> id, according to their respective 
group, either a multiple-choice or a short answer test. 
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' Meanwhile, after wlal^^^ng. th^t -the coordinator of the educational 
psycholcgy classes was working withliha studpnts wl^ had left the room, the 
instructor- in the regular classroom routinely 'administered from one xommon ^ 
stack either a multiple-choice or a short answer test .respectively to 
Students in-Group 1— told test-multiple-choice and Group 2— told test- 
short~answer. At the beginning of the next class meeting, ^students in^ 
Groups l^and 2 -were^irvformed, as their classmates had been earlier, that ' • 
their class- was 'participating in an evaluation of instructional procedures 
which would be explained further at a later date. 

Six weeks Tater, during which time the regular^ scheduled classroom- . 

activities enSued, an experimenter came .back to each dlass on an unannounced 
.basis and adm-inistered both experimental tests to all students. They were 

told that their performance on these tests was the cone lauding part, of the 
"evaluation they had earlier been asked to participate in. Each person was 

Strongly 'encouraged to do his or her best on both tests with an added 

incentive for conscientious effort being an "AVinstead of the unit test 

V ■ . 

scores. / 

Data Preparation and Analysjs . In order to guard against bias in 
the scoring of the test, all identifying invormation was concealed and each 
test was assigned an identifying number. The multiple-choice testswere 
then scored using, an oi)jective scoring key. A detailed scoring key was 
prepared and used to score the short answer tests. In order to have 
uniformity throughout, one experimenter^with the freviously demonstrated, 
ability to reliably score short answer t;ests (r > .98) scored all tests. 

ResuTtiny scores from the retention tests were analyzed using a 
2X3 X'2 X 2 factoriaT analysis of variance with repeatejl measures on- 



the last'two factors^ T lie- two between factors were study -condition (test, ^ 
no test) and test treatment condition (muUi pi e-ctioice, short answer, no ' 
"test)>--The within factors were item type on criterion measure (multiple^ 
choice, short answer) and item complexity within the criterion, test (knowledge. . 
concept). /For all significant interactions and.for appropriate main effects 
Tuckey's HSD test was used for making pairwise. coi^jpartsons' of the means. All 
effects reported as ^"ignificant were at' the .05 level or less. 

Results 

, ; The main effect for study condi''t ion was significant, £ (1, 84)> = 11.65, 
£< .001. .Subjects in the study condition-test group (X = 8,79) scored: 
higher than .subjects in the study condition-no test group (X ='7.53). (Note: 
All means reported ane an average of the means for levels of factoYs involved.) 

■ 4 

The main effect for the test treatment condition was significant, F. (2, 84) = 
4.95*, £< .01. Although differences did exist among the multiple-choice 
treatment group (X = 8.825), the' short-answer treatment group (X = 8.242), 
and the no test treatment group (X = 7.048). these differences were not ^explored 
due to the significant interaction between study condition and test treatment 
condition, F (2, 84) = 6.50, i < .01. Results of Tuckey's test revealed 
that stu^- condition test' subjects who took eitjier the multiple-choice 
test (X = 9.58) or the short-answer test (X = 9.62) scored higher than 
subjects who took no test (X = 7.17), but the means for the multiple-choice 
test up and the, short-answer test g>'oup did not significantly differ. 

,e study condition-no test, however, means for subjects in the 
mu.ciple-choice' (X - 8.07), short-answer (X = 6.87) and no test (X = 7,65) 
treatment groups did not differ significantly. 



Although there was a significant main effect for the first within group 
factor, criterion, item type. F (1, 84)' = 328.75, ^ < yOl, knowing that students 
scof-e higher on multiple-choice items than on short- answer ^it4ms contributes 
'little or no valuable information. However, the significant interaction * 
between the within group factor item type (mul tiple-chpice vs. short answer) 
and the betwfeeri group factor test treatment condition (multiple-cho4ce, 
shoH answer, and- no test) is of importance, F (2, 84) = 3.38', £ < -05. . . 
A^ain using Tuckey's HSD test, on the multiple-choice dependent\easure 
subjects who took the multiple-choice test initially as a treatment (multipTe- 
ch(/ice treatment group) (X = 11.08) scored higher thai\- subjects who took the"- 
short-answer test as a treatment (short-answer tesH; treatment group) (X 9.85) 
and the subjects who received no test at the initial testing* time (no test 
treatment group) (X = 9.58). The short-answer test treatment group failed " 
to differ from the no test treatment' group. Using the response. to the short- 
answer questions as the dependent measure, a different pattern of subject 
responses was found.- Subjects in the no test treatment group (X = 5.23) 
scored lower than subjects in the multiple-choice treatment group (X = 6.57) 
and subjects fn the short-answer treatment group- (X" = 6.63). - Subjects in 
the latter two groups failed to difYer, 

Criterion item type failed to interact with the between group factor of 
study condition, 

With the final withi-n group factor, item complexity, there was a 
significant main effect, F (1, 84) = 11.07, £ < .01. Item complexity did not 
' interact with either of the between group factors, test treatment condition and 
study condition. There was a significant interaction between the two within, 
group factors, item complexity and item type, £ (1, 84) = 59.45, ^ < .001. 



However, finding that students , scored higher.^on conceptual questions c&st 
in a multiple-choice format but l%er on conceptual items cast in a short- 
answer type item format was of, minimum value to the study. 

Nome of the three- and four-way interacjtions were significant. 

' ' Conclusions 
The most profound conclusion supported 'by the results of this study 
is that students should both study for and take a test to maximize retention. 
Such a conclusion is diametrically opposed to observations such as thiDse by 

< t » 

John Holt (1964): "the test-examination-marks business is a gigantic racket, 
the purpose of which is to enable students, teachers, and schools to take 
part in a joint pretense" • (p. 13j). It certainly is not in line with 
recommendations such as those of Postman and Weingartner (1969)' which call 
for a moratoriiini on the use of textbooks-, eliminatiorf of all courses and 

7 • 

all course requirements, and the abolition of tests and yrades. 

A secondary coriclus'ion supported in this study js that, even though 
testing seems to influence retention, this influence is in part a joint 
function not only of the type of test initially administered^- but also the 
type of test used^as a criterion measure., , 



s Summary Referepces ' . . '' 
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Figure 1. Interaction of'Te^t Condition and Study Condition. 
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t) • ' TABLE 1 . . 

MOVA OF STUDY CONDITION. TEST CONDITION. IT£M TYPE. AND 
ITtM COMPLEXITY FOR RETENTION TEST . I 



Source 



Study Condition 

test Condition / 
Study Condition X Test Condition 
Between Group Factor •'. 



Item Type 

Item fype X Study Condition 
Itvjm Type )/Test Condition 
Item Type X- Study Condition 
Viithin Group Error (1) 



X T«st Condition 



Item Complexity 

.Item Complexity X Study Condition 
Item Cdtnplexity X Test Condit^ion 
Item Complexity X Study Condition X'Test Condition 
Within Group Error (2) 



I tern Type X I tem. Compl ex i ty 
Item Type-X Iteqj Complexity X 
Item Type X Item Complexity X 
Item Type" X Item Complexity X 
Within Group Error (3) 



Study Condition 
Test Condition 
Study Condition 



\ Test Condition 



***£ < . 001 



**£ < .01 





■ ^ < 




Sum of 


Degrees of 


F 




Freedom . 


Vslue . 


143.14 


. 1 ^ 


H.65**t 


1 ?r 67 


Z ^ 


4^95**V 


159 75 ^ 


- 2 \ V 6.50** \ 


1 031 66 


84 




1460 05 


■ 1 


328.75***' 


, 7.80 


1 ■ ' 


•1.76 


30.02 


2 


3,38*' - 


12.29 ■ 


2 • 


}.38 . 


37l06 


84 




-40-67 


1 


ll.Q7f** 


• 0.62 


1 


0.17, 


2.Q2 


. a 


. 0.28 * 


13.40 


2 


1.82 • 


3Q8.53 


84, 
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• 95-07 


1 


59.45*^* 


1.00 


1 


0.63 


5.76 
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• 1.80 


1.09 
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134.33 
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TABLE 2 



■ • RETENTION TEST MEANS AND STANDARD DEVIATIONS FOR TREATMENT GROUPS 



Retention 
Test 



Multiple-Choice 
Knowl edge 

Multiple-Choice 
Concept 

Short Answer 
Knowledge 

Short Answer 
Concept 



Prepared 
Multiple- 
Choice 
Mean SD 



Prepared 
Short 
Answer 
Mean SO 



Prepared 
No 

Test 
Mean SD 



Not 
Prepared 
Multiple- 
Choice 
Mean SD 



Not 
Prepared 
Short 
Answer 
Mean SD 



Not 
Prepared 

No , 
Test 
Mean SD 



10.87 1.81 9.60 2.23 8.67 2.8^ 9.60 1.72 8.27 2.63 8.93 2.09 

12.47 1.68 12.13 1.96 10.20 2.37 11.40 1.76 9.40 2.16 ^10.53 2.45 

7.80 2.60 8 47 1.77 5.07 1.83 .5.37' 2.26 5.63 1.92 5.20 2.54 

7.20 3.12 8.27 2.79 4.73 2.66 5.40 2,59 4.27 1.16 5.93 3.65 
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